Automatic processing chain generation

ABSTRACT

Disclosed herein is a method for designing a processing chain of a sensor system, the method comprising receiving a desired application comprising at least one activity for a sensor system to monitor, the sensor system comprising at least one sensor capable of generating sensor data based on sensing the at least one activity, accessing a database comprising a plurality of raw sensor data and a plurality of annotations corresponding to the plurality of raw sensor data, the plurality of annotations identifying activities corresponding to the plurality of raw sensor data, and automatically generating a processing chain of the sensor system for executing the desired application based on the desired application and the plurality of raw sensor data, the processing chain for processing the sensor data and for extracting at least one feature from the sensor data for use in sensing the at least one activity.

RELATED APPLICATION

This application claims priority to and the benefit of co-pending U.S.Provisional Patent Application 63/260,660, filed on Aug. 27, 2021,entitled “AI FRAMEWORK FOR SENSOR SYSTEMS,” by Abbas Ataya, havingAttorney Docket No. IVS-1003-PR, and assigned to the assignee of thepresent application, which is incorporated herein by reference in itsentirety

BACKGROUND

Mobile electronic devices often have limited resources for computingability, so it is beneficial to design systems within the mobile deviceto be efficient. Processors with compilers tend to require morerecourses than processors without compilers, such as battery life,memory, performance, etc. When using compilers, libraries for thecompiler are required, and can result in slower computation speeds.

Decision node trees are built to account for as many decisions orscenarios that the developer may view as being relevant. However, asingle user is unlikely to fully utilize every option of the decisiontree. A way to streamline the decision tree would be beneficial as itwould save on memory, processing power, and battery life.

When pre-processing data to be used in applications, human input istypically required to determine what pre-processing methods should beapplied to the data to make it more closely resemble the expectedphenomenon. The past experience and intuition of the human designer isgenerally the main input into making processing chains and understandingthe system limitations.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthe Description of Embodiments, illustrate various embodiments of thesubject matter and, together with the Description of Embodiments, serveto explain principles of the subject matter discussed below. Unlessspecifically noted, the drawings referred to in this Brief Descriptionof Drawings should be understood as not being drawn to scale. Herein,like items are labeled with like item numbers.

FIG. 1 illustrates a block diagram of an example electronic device andtraining machine.

FIG. 2 illustrates a block diagram of the machine learning engine flow,according to some embodiments.

FIG. 3 illustrates a block diagram of the deployed model according tosome embodiments.

FIG. 4 illustrates a flow diagram of an example process for a dataprocessing method, according to some embodiments.

FIG. 5 illustrates a diagram of a decision tree model customizationsystem, according to some embodiments.

FIG. 6 illustrates a flow diagram of an example process for modifying atrained classification model, according to some embodiments.

FIG. 7 illustrates an example output of the resource optimizer in theform of a table showing importance and cost of each feature, accordingto some embodiments.

FIG. 8 illustrates an example process for designing a processing chainof a sensor system, according to some embodiments.

DESCRIPTION OF EMBODIMENTS

The following Description of Embodiments is merely provided by way ofexample and not of limitation. Furthermore, there is no intention to bebound by any expressed or implied theory presented in the precedingbackground or brief summary, or in the following detailed description.

Reference will now be made in detail to various embodiments of thesubject matter, examples of which are illustrated in the accompanyingdrawings. While various embodiments are discussed herein, it will beunderstood that they are not intended to limit to these embodiments. Onthe contrary, the presented embodiments are intended to coveralternatives, modifications and equivalents, which may be includedwithin the spirit and scope the various embodiments as defined by theappended claims. Furthermore, in this Description of Embodiments,numerous specific details are set forth in order to provide a thoroughunderstanding of embodiments of the present subject matter. However,embodiments may be practiced without these specific details. In otherinstances, well known methods, procedures, components, and circuits havenot been described in detail as not to unnecessarily obscure aspects ofthe described embodiments.

Notation and Nomenclature

Some portions of the detailed descriptions which follow are presented interms of procedures, logic blocks, processing and other symbolicrepresentations of operations on data within an electrical circuit.These descriptions and representations are the means used by thoseskilled in the data processing arts to most effectively convey thesubstance of their work to others skilled in the art. In the presentapplication, a procedure, logic block, process, or the like, isconceived to be one or more self-consistent procedures or instructionsleading to a desired result. The procedures are those requiring physicalmanipulations of physical quantities. Usually, although not necessarily,these quantities take the form of electrical or magnetic signals capableof being stored, transferred, combined, compared, and otherwisemanipulated in an electronic device.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the followingdiscussions, it is appreciated that throughout the description ofembodiments, discussions utilizing terms such as “receiving,”“performing,” “generating,” “selecting,” “adjusting,” “comparing,”“prioritizing,” “modifying,” “adding,” associating,” “filtering,”“updating,” “forwarding,” “labeling,” or the like, refer to the actionsand processes of an electronic device such as an electrical circuit.

Embodiments described herein may be discussed in the general context ofprocessor-executable instructions residing on some form ofnon-transitory processor-readable medium, such as program modules,executed by one or more computers or other devices. Generally, programmodules include routines, programs, objects, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. The functionality of the program modules may becombined or distributed as desired in various embodiments.

In the figures, a single block may be described as performing a functionor functions; however, in actual practice, the function or functionsperformed by that block may be performed in a single component or acrossmultiple components, and/or may be performed using hardware, usingsoftware, or using a combination of hardware and software. To clearlyillustrate this interchangeability of hardware and software, variousillustrative components, blocks, modules, logic, circuits, and stepshave been described generally in terms of their functionality. Whethersuch functionality is implemented as hardware or software depends uponthe particular application and design constraints imposed on the overallsystem. Skilled artisans may implement the described functionality invarying ways for each particular application, but such implementationdecisions should not be interpreted as causing a departure from thescope of the present disclosure. Also, the example mobile electronicdevice described herein may include components other than those shown,including well-known components.

Various techniques described herein may be implemented in hardware,software, firmware, or any combination thereof, unless specificallydescribed as being implemented in a specific manner. Any featuresdescribed as modules or components may also be implemented together inan integrated logic device or separately as discrete but interoperablelogic devices. If implemented in software, the techniques may berealized at least in part by a non-transitory processor-readable storagemedium comprising instructions that, when executed, perform one or moreof the methods described herein. The non-transitory processor-readabledata storage medium may form part of a computer program product, whichmay include packaging materials.

The non-transitory processor-readable storage medium may comprise randomaccess memory (RAM) such as synchronous dynamic random access memory(SDRAM), read only memory (ROM), non-volatile random access memory(NVRAM), electrically erasable programmable read-only memory (EEPROM),FLASH memory, other known storage media, and the like. The techniquesadditionally, or alternatively, may be realized at least in part by aprocessor-readable communication medium that carries or communicatescode in the form of instructions or data structures and that can beaccessed, read, and/or executed by a computer or other processor.

Various embodiments described herein may be executed by one or moreprocessors, such as one or more motion processing units (MPUs), sensorprocessing units (SPUs), host processor(s) or core(s) thereof, digitalsignal processors (DSPs), general purpose microprocessors, applicationspecific integrated circuits (ASICs), application specific instructionset processors (ASIPs), field programmable gate arrays (FPGAs), aprogrammable logic controller (PLC), a complex programmable logic device(CPLD), a discrete gate or transistor logic, discrete hardwarecomponents, or any combination thereof designed to perform the functionsdescribed herein, or other equivalent integrated or discrete logiccircuitry. The term “training processor,” as used herein may refer toany of the foregoing structures or any other structure suitable forimplementation of the techniques described herein. As it employed in thesubject specification, the term “training processor” can refer tosubstantially any computing processing unit or device comprising, butnot limited to comprising, single-core processors; single-processorswith software multithread execution capability; multi-core processors;multi-core processors with software multithread execution capability;multi-core processors with hardware multithread technology; parallelplatforms; and parallel platforms with distributed shared memory.Moreover, processors can exploit nano-scale architectures such as, butnot limited to, molecular and quantum-dot based transistors, switchesand gates, in order to optimize space usage or enhance performance ofuser equipment. A processor may also be implemented as a combination ofcomputing processing units. Any code trained on the “training processor”may then be deployed on a reduced instruction set computer, or an“reduced instruction set computer (RISC) processor.”

In addition, in some aspects, the functionality described herein may beprovided within dedicated software modules or hardware modulesconfigured as described herein. Also, the techniques could be fullyimplemented in one or more circuits or logic elements. A general purposeprocessor may be a microprocessor, but in the alternative, the processormay be any conventional processor, controller, microcontroller, or statemachine. A processor may also be implemented as a combination ofcomputing devices, e.g., a combination of an SPU/MPU and amicroprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with an SPU core, MPU core, or any othersuch configuration.

Overview of Discussion

Discussion begins with a description of an example electronic devicewith which or upon which various embodiments described herein may beimplemented. Examples of an electronic device and training machine arethen described. Examples of the training phase and the deployment phaseare then described. Examples of Model Customization are then described.Examples of Automatic Processing Chain Generation are then described.

Electronic mobile devices, in accordance with the described embodiments,may be used for a variety of purposes. In some cases, there are needsfor the electronic mobile device to monitor a user and their activities(e.g., walking, running, sleeping, etc.). In such cases, it is usefulfor the electronic mobile device to have an efficient system that mayadapt to each user.

Embodiments described herein receive data, process the data at a fixedcode processing engine, wherein operation of the fixed code processingengine is controlled according to stored parameters, then classify theprocessed data at a fixed code classification engine, wherein operationof the fixed code classification engine is controlled according to thestored parameters.

Embodiments described herein provide an electronic device including atleast one sensor device, a processor, and a memory device havingprocessor-executable code stored thereon for execution by the processor.The code includes a fixed code processing engine for processing sensordata received from the at least one sensor device, a fixed codeclassification engine for classifying processed sensor data, where thefixed code processing engine and the fixed code classification engineare configurable according to a plurality of parameters that defineoperation of the fixed code processing engine and the fixed codeclassification engine.

Embodiments described herein include a method for modifying a trainedclassification model, the method including receiving feature dataextracted from sensor data, classifying the feature data according tothe trained classification model to identify a label corresponding tothe feature data, wherein the trained classification model includes adecision tree including a plurality of decision nodes for featureidentification fora plurality of features, tracking identified featuresof the plurality of features over a predetermined amount of time, andresponsive to determining that a feature of the trained classificationmodel does not satisfy a frequency of usage threshold over thepredetermined amount of time, deactivating a decision node of thedecision tree of the trained classification model corresponding to thefeature such that a subsequent instance of the classifying does notconsider a deactivated decision nodes for subsequently received featuredata.

Embodiments described herein include a method for designing a processingchain of a sensor system, the method comprising receiving a desiredapplication comprising at least one activity for a sensor system tomonitor, the sensor system comprising at least one sensor capable ofgenerating sensor data based on sensing the at least one activity,accessing a database comprising a plurality of raw sensor data and aplurality of annotations corresponding to the plurality of raw sensordata, the plurality of annotations identifying activities correspondingto the plurality of raw sensor data, and automatically generating aprocessing chain of the sensor system for executing the desiredapplication based on the desired application and the plurality of rawsensor data, the processing chain for processing the sensor data and forextracting at least one feature from the sensor data for use in sensingthe at least one activity.

Example Electronic Device and Training Machine

Turning now to the figures, FIG. 1 illustrates a block diagram of anexample electronic device 100 and training machine 101, according tosome embodiments. As will be appreciated, electronic device 100 may beimplemented as a device or apparatus, such as a handheld mobileelectronic device. For example, such a mobile electronic device may be,without limitation, a mobile telephone phone (e.g., smartphone, cellularphone, a cordless phone running on a local network, or any othercordless telephone handset), a wired telephone (e.g., a phone attachedby a wire), a personal digital assistant (PDA), a video game player,video game controller, a Head Mounted Display (HMD), a virtual oraugmented reality device, a navigation device, an activity or fitnesstracker device (e.g., bracelet, clip, band, or pendant), a smart watchor other wearable device, a mobile internet device (MID), a personalnavigation device (PND), a digital still camera, a digital video camera,a portable music player, a portable video player, a portable multi-mediaplayer, a remote control, or a combination of one or more of thesedevices. In other embodiments, electronic device 100 may be implementedas a fixed electronic device, such as and without limitation, anelectronic lock, a doorknob, a car start button, an automated tellermachine (ATM), etc. Training machine 101 may be implemented as a deviceor apparatus, for example a computer, such as a desktop computer, serverrack, virtual machine, laptop, etc.

As depicted in FIG. 1 , training machine 101 may include a hostprocessor 110, a host bus 120, a host memory 130, and a sensorprocessing unit 170. Some embodiments of training machine 101 mayfurther include one or more of a display device 140, an interface 150, atransceiver 160 (all depicted in dashed lines) and/or other components.In various embodiments, electrical power for training machine 101 isprovided by a mobile power source such as a battery (not shown), whennot being actively charged.

Host processor 110 can be one or more microprocessors, centralprocessing units (CPUs), DSPs, general purpose microprocessors, ASICs,ASIPs, FPGAs or other processors which run software programs orapplications, which may be stored in host memory 130, associated withthe functions and capabilities of training machine 101.

Host bus 120 may be any suitable bus or interface to include, withoutlimitation, a peripheral component interconnect express (PCIe) bus, auniversal serial bus (USB), a universal asynchronousreceiver/transmitter (UART) serial bus, a suitable advancedmicrocontroller bus architecture (AMBA) interface, an Inter-IntegratedCircuit (I2C) bus, a serial digital input output (SDIO) bus, a serialperipheral interface (SPI) or other equivalent. In the embodiment shown,host processor 110, host memory 130, display 140, interface 150,transceiver 160, sensor processing unit (SPU) 170, and other componentsof training machine 101 may be coupled communicatively through host bus120 in order to exchange commands and data. Depending on thearchitecture, different bus configurations may be employed as desired.For example, additional buses may be used to couple the variouscomponents of training machine 101, such as by using a dedicated busbetween host processor 110 and memory 130.

Host memory 130 can be any suitable type of memory, including but notlimited to electronic memory (e.g., read only memory (ROM), randomaccess memory, or other electronic memory), hard disk, optical disk, orsome combination thereof. Multiple layers of software can be stored inhost memory 130 for use with/operation upon host processor 110. Forexample, an operating system layer can be provided for training machine101 to control and manage system resources in real time, enablefunctions of application software and other layers, and interfaceapplication programs with other software and functions of trainingmachine 101. Similarly, a user experience system layer may operate uponor be facilitated by the operating system. The user experience systemmay comprise one or more software application programs such as menunavigation software, games, device function control, gesturerecognition, image processing or adjusting, voice recognition,navigation software, communications software (such as telephony orwireless local area network (WLAN) software), and/or any of a widevariety of other software and functional interfaces for interaction withthe user can be provided. In some embodiments, multiple differentapplications can be provided on a single training machine 101, and insome of those embodiments, multiple applications can run simultaneouslyas part of the user experience system. In some embodiments, the userexperience system, operating system, and/or the host processor 110 mayoperate in a low-power mode (e.g., a sleep mode) where very fewinstructions are processed. Such a low-power mode may utilize only asmall fraction of the processing power of a full-power mode (e.g., anawake mode) of the host processor 110.

Display 140, when included, may be a liquid crystal device, (organic)light emitting diode device, or other display device suitable forcreating and visibly depicting graphic images and/or alphanumericcharacters recognizable to a user. Display 140 may be configured tooutput images viewable by the user and may additionally or alternativelyfunction as a viewfinder for camera. It should be appreciated thatdisplay 140 is optional, as various electronic devices, such aselectronic locks, doorknobs, car start buttons, etc., may not require adisplay device.

Interface 150, when included, can be any of a variety of differentdevices providing input and/or output to a user, such as audio speakers,touch screen, real or virtual buttons, joystick, slider, knob, printer,scanner, computer network I/O device, other connected peripherals andthe like.

Transceiver 160, when included, may be one or more of a wired orwireless transceiver which facilitates receipt of data at trainingmachine 101 from an external transmission source and transmission ofdata from training machine 101 to an external recipient. By way ofexample, and not of limitation, in various embodiments, transceiver 160comprises one or more of: a cellular transceiver, a wireless local areanetwork transceiver (e.g., a transceiver compliant with one or moreInstitute of Electrical and Electronics Engineers (IEEE) 802.11specifications for wireless local area network communication), awireless personal area network transceiver (e.g., a transceivercompliant with one or more IEEE 802.15 specifications for wirelesspersonal area network communication), and a wired a serial transceiver(e.g., a universal serial bus for wired communication).

In some embodiments, electronic device 100 includes a display (notshown) similar to display 140. In some embodiments, electronic device100 includes an interface (not shown) similar to interface 150. In someembodiments, electronic device 100 includes a transceiver (not shown)similar to transceiver 160.

As depicted in FIG. 1 , electronic device 100 may include a generalpurpose sensor assembly in the form of integrated Sensor Processing Unit(SPU) 170 which includes sensor processor 172, memory 176, a motionsensor 178, and a bus 174 for facilitating communication between theseand other components of SPU 170. In some embodiments, SPU 170 mayinclude at least one additional sensor 180 (shown as sensor 180-1,180-2, . . . 180-n) communicatively coupled to bus 174. In someembodiments, all of the components illustrated in SPU 170 may beembodied on a single integrated circuit. It should be appreciated thatSPU 170 may be manufactured as a stand-alone unit (e.g., an integratedcircuit), that may exist separately from a larger electronic device andis coupled to a host bus through an interface (not shown).

Some embodiments of electronic device 100 may further include one ormore of a display device, an interface, a transceiver (all not shown)and/or other components. In various embodiments, electrical power forelectronic device 100 is provided by a mobile power source such as abattery (not shown), when not being actively charged.

Sensor processor 172 can be one or more microprocessors, CPUs, DSPs,general purpose microprocessors, ASICs, ASIPs, FPGAs or other processorswhich run software programs, which may be stored in memory 176,associated with the functions of SPU 170. It should also be appreciatedthat motion sensor 178 and additional sensor 180, when included, mayalso utilize processing and memory provided by other components ofelectronic device 100.

In some embodiments, sensor processor 172 is a reduced instruction setcomputer (RISC). In this embodiment, there is an internal measurementunit which is run on the sensor processor 172.

Bus 174 may be any suitable bus or interface to include, withoutlimitation, a peripheral component interconnect express (PCIe) bus, auniversal serial bus (USB), a universal asynchronousreceiver/transmitter (UART) serial bus, a suitable advancedmicrocontroller bus architecture (AMBA) interface, an Inter-IntegratedCircuit (I2C) bus, a serial digital input output (SDIO) bus, a serialperipheral interface (SPI) or other equivalent. Depending on thearchitecture, different bus configurations may be employed as desired.In the embodiment shown, sensor processor 172, memory 176, motion sensor178, and other components of SPU 170 may be communicatively coupledthrough bus 174 in order to exchange data.

Memory 176 can be any suitable type of memory, including but not limitedto electronic memory (e.g., read only memory (ROM), random accessmemory, or other electronic memory). Memory 176 may store algorithms orroutines or other instructions for processing data received from motionsensor 178 and/or one or more sensor 180, as well as the received dataeither in its raw form or after some processing. Such algorithms androutines may be implemented by sensor processor 172 and/or by logic orprocessing capabilities included in motion sensor 178 and/or sensor 180.

Motion sensor 178 is a sensor capable of sensing a form or type ofmotion, including without limitation a gyroscope or an accelerometer. Asensor 180 may comprise, without limitation: a temperature sensor, ahumidity sensor, an atmospheric pressure sensor, an infrared sensor, aradio frequency sensor, a navigation satellite system sensor (such as aglobal positioning system receiver), an acoustic sensor (e.g., amicrophone), another inertial or motion sensor (e.g., a gyroscope,accelerometer, or magnetometer) for measuring the orientation or motionof the sensor in space, or other type of sensor for measuring otherphysical or environmental conditions. In one example, sensor 180-1 maycomprise an acoustic sensor, sensor 180-2 may comprise a temperaturesensor, and sensor 180-n may comprise a motion sensor.

In some embodiments, motion sensor 178 and/or one or more sensors 180may be implemented using a microelectromechanical system (MEMS) that isintegrated with sensor processor 172 and one or more other components ofSPU 170 in a single chip or package. Although depicted as being includedwithin SPU 170, one, some, or all of motion sensor 178 and/or one ormore sensors 180 may be disposed externally to SPU 170 in variousembodiments.

Examples of Training Phase and Deployment Phase

FIG. 2 illustrates a block diagram of the machine learning engine flow200, according to some embodiments. Training phase 202 takes place ontraining machine 101. Once the training phase 202 is complete, thetrained machine learning (ML) classifier 208 is deployed on electronicdevice 100 in the deployment phase 204.

The training phase 202 revolves around training a ML classifier 208 tocorrectly identify patterns, and to make decisions from previous data.For example, if there is a large amount of data on a walking motion anda running motion, then after training the ML classifier 208 can receivea new set of data and correctly classify the new data as either walkingor running. The data used for training may vary in the features includedsuch that different models may be trained and deployed for different usecases.

In some embodiments, the training phase 202 is done offline. In someembodiments, the training phase 202 is not restricted in memory, speed,cost, etc., as the deployment phase 204 might be.

In the training phase 202, a labeled database 206 is used to train theML classifier 208. The data within the labeled database 206 containsdata that will undergo preprocessing to extract features, the extractedfeatures then being input into the ML classifier which should be able toaccurately classify and label the features. Features may include mean,variance, energy, number of peaks, peak distance, mean cross rate,dominant frequencies, spectral power, etc. The data within the labeleddatabase 206 contains corresponding labels to what activity the datadescribes. For example, if the data is a set of motion data from aperson running, then the data will be labeled as running. Otheractivities may include walking, sleeping, sitting, driving, exercising,etc.

First, the data will go through pre-processing 210. This stage mayinclude eliminating noise in the data and putting the data in a moreappropriate format for the following steps. Next, the data will gothrough the feature extraction 212 stage. Here, the data is analyzed(without the labels) for what activity the data might portray. Next, inthe feature selection 214 stage, the less important features arefiltered out, and the more important features are retained. The featureselection 214 may be done through a numerical threshold, human input,redundancy, or similar filtering methods.

Finally, the selected features are put into the ML classifier 208, whichwill take the features and label them. The ML classifier 208 labels maythen be compared against the labels from the labeled database 206 foraccuracy.

The training phase 202 is repeated until the ML classifier 208 is ableto accurately label the features to a satisfactory degree. During theserepetitions, ML classifier 208 is adjusted with the goal of being ableto accurately label features. Once the ML classifier is judged to beready, the deployment phase 204 is started.

Deployment phase 204 has the ML classifier 208 deployed on an electronicdevice (e.g., electronic device 100). In some embodiments, ML classifier208 is deployed as a fixed code classification engine stored on memory176 for execution by sensor processor 172. Once deployed, data sample216 becomes the new input. In some embodiments, data sample 216 isreceived from at least motion sensor 178 or sensor 180. In someembodiments, data sample is a mix of preprocessed data and real timedata.

After being received, data sample 216 will go through preprocessing 218where noise is removed, and the data is formatted. Next, based on thefeature extraction 212 and feature selection 214 stages of trainingphase 202, the data will go through the selected features extraction 220phase. Here, the features that were determined to be important duringthe training phase 202 are extracted from the data. The extractedfeatures will then go through ML classifier 208, where they receive alabel 222 according to the activity performed in the feature (e.g.,walking, running, sleeping, etc.).

In some embodiments, the described embodiments are run on a processorwithout a compiler during the deployment phase 204. Traditional systemswith compilers require hard coded libraries for the processes andtranslation to machine language, which can slow down whatever process isbeing run. By using a processor without a compiler, such as a RISCprocessor, aspects such as performance, battery life, and cost areimproved over general purpose processors.

FIG. 3 illustrates a block diagram of the deployed model 300. Thedeployed model has an input of data 316. Data 316 is run throughprocessing engine 324, before being run though ML classifier 308.

In some embodiments, the deployed model 300 is deployed on a processorthat does not have a compiler. The benefits of running on a processorwith no compiler include being cost efficient, power efficient, and highperformance. As the fixed code processing engine and the fixed codeclassification engine are fixed in the memory for execution by aprocessor without a compiler, the described embodiments are ideal forsuch a system due to its ability to edit the function with parameters.

Data 316 may also be referred to as input data, sensor data, or gathereddata. In some embodiments, data 316 is collected from motion sensor 178.In some embodiments, data 316 is collected from sensors 180. In someembodiments, processing engine 324 is a fixed code processing engine324. In some embodiments, ML classifier 308 is a fixed codeclassification engine 308.

In the processing engine 324 data 316 is filtered and features areextracted. The filtering of the sensor data and extraction of thefeatures are controlled by computation engine parameters 326.Computation engine parameters 326 are updatable. In some embodiments,the computation engine also segments the data 316 into windows of anumber of samples.

After passing through the processing engine 324, the processed sensordata is passed to the classification engine 308 where the data isclassified. In some embodiments, the classification engine 308determines and labels the processed sensor data.

In some embodiments, classification engine parameters 328 control theoperation of classification engine 308. The parameters can affect theclassification and labeling of the processed sensor data. Classificationengine parameters 328 are updatable.

Classification engine parameters 328 and computation engine parameters326 may collectively be referred to as the stored parameters. In someembodiments, an updated set of parameters is received, and the storedparameters are updated according to the updated parameters. In someembodiments, the stored parameters can automatically update based on theprocessed sensor data.

In some embodiments, the stored parameters can enable or disable afunction of the processing engine 324. In some embodiments, the storedparameters can enable or disable a function of the classification engine308. In some embodiments, the stored parameters are stored in the RAM.In some embodiments, the stored parameters can be manually updated. Thestored parameters may also be referred to as a plurality of parameters.

FIG. 4 illustrates a flow diagram 400 of an example process for a dataprocessing , according to some embodiments. Procedures of these methodswill be described with reference to elements and/or components ofvarious figures described herein. It is appreciated that in someembodiments, the procedures may be performed in a different order thandescribed, that some of the described procedures may not be performed,and/or that one or more additional procedures to those described may beperformed. The flow diagrams include some procedures that, in variousembodiments, are carried out by one or more processors (e.g., a hostprocessor or a sensor processor) under the control of computer-readableand computer-executable instructions that are stored on non-transitorycomputer-readable storage media. It is further appreciated that one ormore procedures described in the flow diagrams may be implemented inhardware, or a combination of hardware with firmware and/or software.

At procedure 410 of flow diagram 400, data is received by the system. Insome embodiments, the data is received from a sensor that collected thedata.

At procedure 420, the data is processed at the fixed code processingengine, according to the stored parameters. Here, the data is filtered(e.g., for noise), and the features are extracted. The extractedfeatures are determined based on the stored parameters. In oneembodiment, the processing engine utilizes the computation engineparameters to control the operation of the processing engine. In oneembodiment, the computation engine parameters control the filtering ofdata and extracting the features from the data. After this step, thedata is referred to as processed data.

At procedure 430, the processed data is classified in the fixed codeclassification engine, according to the stored parameters. In oneembodiment, the classification engine utilizes the classification engineparameters to control the operation of the classification engine. In oneembodiment, the classification engine parameters control theclassification and labeling of the processed data for a plurality ofapplications.

At procedure 440, after the processed data is classified, theclassification engine determines and assigns a label to the data.

In some embodiments, at procedure 450, an updated set of parameters isreceived. In some embodiments this updated set of parameters is manuallyupdated. In some embodiments the updated set of parameters areautomatically generated.

In some embodiments, at procedure 460, the stored parameters are updatedaccording to the updated set of parameters.

In some embodiments, the stored parameters comprise the computationengine parameters and the classification engine parameters. In someembodiments, the stored parameters can enable and disable a function ofthe fixed code processing engine. In some embodiments, the storedparameters can enable and disable a function of the fixed codeclassification engine.

Examples of Model Customization

FIG. 5 illustrates a diagram of a decision tree model customization 500system, according to some embodiments. Shown in the figure is theoffline training phase 530, generic model 532, the on-chip deploymentphase 534, and the adapted model 536.

The system shown in at least FIG. 5 is capable of adapting to anindividual user and their habits such that the device used in theon-chip deployment phase may be more efficient.

Offline training phase 530 takes place on a computer similar to that oftraining machine 101. In some embodiments, offline training phase takesplace on a virtual machine. During offline training phase 530, a genericmodel 532 is created. In some embodiments generic model 532 is adecision tree comprising a plurality of nodes. In some embodiments,generic model 532 is a trained classification model, wherein the trainedclassification model identifies a label corresponding to the featuredata, wherein the plurality of decision nodes are used for featureidentification for a plurality of features.

In some embodiments the offline training phase 530 is done with machinelearning. In some embodiments, offline training phase 530 is similar instructure to training phase 202.

Once the generic model 532 is determined to be of suitable quality, itis then moved to the on-chip deployment phase 534. In some embodiments,the on-chip deployment phase 534 takes place on a mobile electronicdevice similar to that of electronic device 100. In some embodiments,the on-chip deployment phase takes place on stationary electronicdevices (for example, fitness equipment, household appliances, etc.).

On the first day of usage, the deployed model will be the same as thegeneric model. Over time, the model will disable nodes and paths in thedecision tree that are rarely or never used, which results in adaptedmodel 536.

Benefits of this adaptive system include freeing up memory, increasingprocessing speed, increasing battery life, etc.

In some embodiments, the nodes and branches are initially assigned aweight (or usage frequency) of zero. Each time a node or branch (i.e., afeature) is used, the weight of the feature is increased. This methodwill allow the system to keep track of which features are used mostoften, and which features are used rarely or if at all.

In some embodiments, the weight is checked periodically. In someembodiments, an input prompts the weights to be checked. In someembodiments, the weight is checked over set intervals of time. In someembodiments, there is an initial amount of time that must pass beforethe weights are regularly checked. In some embodiments, the weights arechecked after a certain number of decisions are made. In someembodiments, the intervals over which the weights are checked steadilyincrease until a maximum interval is reached.

In some embodiments, any features with weights below a set threshold aredeemed low usage and are removed from the decision tree. In someembodiments, any features with weights below a set threshold are deemedlow usage and will be checked less often. If a low usage feature remainsas such, then it will be removed from consideration entirely. In someembodiments, a low usage feature will be set to hibernation. In someembodiments, deactivated decision nodes allow the system to save onaspects such as battery life, processing power, computation load, etc.In some embodiments, once a feature is below a threshold it is no longerprocessed.

In some embodiments, a decision node can be reactivated responsive to areset command. In some embodiments, the reset command is automatic. Insome embodiments, the reset command is a manual input. In someembodiments, a path or branch of the decision tree can be reactivatedresponsive to a reset command.

In some embodiments, the weight starts at a set non-zero number and eachtime a feature is processed the weight decreases. In this embodiment,features with weights above a set threshold are considered low usage.

In some embodiments, a table is used to track each time a feature isprocessed. In some embodiments, the weights of features are compared toone another, and a feature is considered low usage if the weight is aset amount lower than an average. In some embodiments, a feature isconsidered low usage if its weight is a statistical outlier whencompares to the other weights. In some embodiments, the threshold is afrequency of usage threshold.

In some embodiments, features commonly used in conjunction are tracked.

In some embodiments, if an intervening feature is determined to be a lowusage feature, then the adapted model may remove that feature from thedecision path while keeping the surrounding features. In someembodiments, new paths can be added between features that are often usedtogether.

In some embodiments, a copy of the generic model 532 is storedseparately in the memory. In this case, if the adapted model isdetermined to be unsatisfactory or no longer necessary (for instance, ifa new person is to take ownership of the device) then the model can bereset. In some embodiments, multiple adapted models are stored formultiple purposes or users. In some embodiments, the multiple adaptedmodels are tied to separate user accounts. For example, if multiplepeople use a single piece of fitness equipment, then the users can havecustomized models on the same piece of equipment.

In some embodiments, the electronic device receives an updated genericmodel 532 with new features. Upon receipt of an updated generic model532, the process is restarted and the adapted model 536 is equivalent tothe updated generic model. The new features would then undergo the sameprocess of determining low use features. In some embodiments, uponreceipt of an updated generic model 532 the frequency with which theweights are tracked is reset.

In some cases, a user might be in a situation where their habits orpatterns change. In some embodiments, after being removed low usagefeatures are periodically checked for potential usage. In someembodiments, removed features can be manually re-activated. In someembodiments, removed features can be automatically re-activated uponrequirement of the feature. In some embodiments, an adapted model can besaved as a backup, or restore state. In some embodiments, a user caninitiate the creation of a backup model. In some embodiments, a user canmanually revert to a backup model. In some embodiments, a backup modelis automatically generated. In some embodiments, an adapted model can beset as a backup model as the generic model is restored.

A threshold may also be referred to as a parameter. Parameters are nothard coded and may be changed or updated. Adjusting the parameters canchange which features or branches are considered in the decision-makingprocess.

In some embodiments, the on-chip deployed model is on a piece ofhardware without a compiler. In these embodiments, parameters would beuseful in allowing the decision tree to adapt without a compiler. Insome embodiments, the on-chip deployed model is on a piece of hardwarewith a compiler.

FIG. 6 illustrates a flow diagram 600 of an example process formodifying a trained classification model, according to some embodiments.Procedures of these methods will be described with reference to elementsand/or components of various figures described herein. It is appreciatedthat in some embodiments, the procedures may be performed in a differentorder than described, that some of the described procedures may not beperformed, and/or that one or more additional procedures to thosedescribed may be performed. The flow diagrams include some proceduresthat, in various embodiments, are carried out by one or more processors(e.g., a host processor or a sensor processor) under the control ofcomputer-readable and computer-executable instructions that are storedon non-transitory computer-readable storage media. It is furtherappreciated that one or more procedures described in the flow diagramsmay be implemented in hardware, or a combination of hardware withfirmware and/or software.

At procedure 610 of flow diagram 600, feature data is received andextracted from sensor data. At procedure 620, the feature data isclassified using the trained classification model in order to identifythe corresponding label. In some embodiments, the trained classificationmodel is the generic model 532. In some embodiments, the trainedclassification model is a decision tree comprising a plurality ofdecision nodes for label identification using a plurality of features.The trained classification model may also be referred to as an adaptedclassification model.

In some embodiments, the identified features of the plurality offeatures are tracked over a predetermined amount of time.

At procedure 630 the usage rate of each feature is tracked over apredetermined amount of time. In some embodiments, for each instance ofan identified feature a weight is applied to the identified feature ofthe plurality of features, and the weights for the plurality of featuresare aggregated over the amount of time.

In some embodiments, each instance of an identified feature has aninitial weight of zero. Each time an instance of an identified featureis used, the weight of the feature is increased.

At procedure 640, the weights for each feature of the plurality offeatures are compared to the frequency of usage threshold over theamount of time, wherein the frequency of usage threshold corresponds toa minimum weight. In some embodiments, the weights used in procedure 640are aggregated weights. In some embodiments, the frequency of usagethreshold is a predetermined value. In some embodiments, the frequencyof usage threshold is dependent on the usage of the device. In someembodiments, the frequency of usage threshold is a variable value.

At procedure 650, responsive to determining that a feature of thetrained classification model does not satisfy a frequency of usagethreshold over the predetermined amount of time, a decision node of thedecision tree of the trained classification model corresponding to thefeature is deactivated, such that a subsequent instance of theclassifying does not consider a deactivated decision nodes forsubsequently received feature data. In some embodiments, subsequentinstance of the classifying that does not consider the deactivateddecision node requires less computational load, power consumption, andmemory than a prior instance of the classifying prior to thedeactivating the decision node.

At procedure 660 a deactivated decision node of the decision tree isreactivated responsive to a reset command. In some embodiments, a resetcommand is a manual input. In some embodiments, a reset commandautomatically occurs after an amount of time.

In some embodiments, the reset command restores the decision tree to itsoriginal state. In some embodiments, the reset command is temporary, andthe decision node will deactivate should it continue to not satisfy afrequency of usage threshold.

In some embodiments, a feature sensing device is implemented for use infeature identification for a user. In some embodiments the featuresensing device is configured for use by a plurality of users, such thateach user of the plurality of users has an associated trainedclassification model and wherein the method is performed separately foreach user of the plurality of users.

In some embodiments, multiple versions of the trained classificationmodel can be stored. In some embodiments, each version of the trainedclassification model is connected to an account, or user profile. Insome embodiments, one instance of the multiple versions of the trainedclassification model can be reset without affecting the other instances.

Examples of Automatic Processing Chain Generation

Determining what features to extract from data, and how to filter thedata to get those features, usually comes from the intuition andknowledge of the designer. One benefit of the current machine learningsystem is that it can automatically build the processing chain withminimal intervention from a user.

In some embodiments, the machine on which the automatic processing chaingeneration takes place on is similar to training machine 101. In someembodiments, the automatic processing chain generation can be segmentedinto the performance optimization stage, and the resource optimizationstage.

The performance optimization stage has the goal of taking theinformation of the desired application and determining which sensors areneeded, which sensor axes are important, what preprocessing should bedone to the raw sensor data, and what set of features will be useful tothe desired application.

To start the performance optimization stage, the system will receive thedesired application information. This is information that outlines whatactivity is to be monitored, and what data should be gathered to monitorthat activity (for instance, monitoring heart rate for exercise, orgyroscope data for walking, etc.).

In some embodiments, the desired application information includeslimitations on the resources the application is allowed to utilize. Insome embodiments, power, memory, and size requirements (or a combinationthereof) are included in the desired application information. In someembodiments, resource limitations are manually added to the desiredapplication information. In some embodiments with resource limitations,the automatic processing chain generator will automatically alter theresults based on the resource limitations. For instance, if a feature orsensor of low importance and high power requirements is included thatgoes above a power limitation, the feature or sensor will automaticallybe disabled.

After receiving the desired application information, the system willthen access the annotated database to retrieve the sensor data andannotations. In some embodiments, the annotated database is similar tolabeled database 206. Annotated database contains the data, and theannotations that note what features instances of data represent. In someembodiments, the data from annotated database is sensor data. In someembodiments, the data in annotated database is timestamped raw sensordata (for example, data from sensors such as an accelerometer,gyroscope, pressure, microphone, etc.) from at least one type of sensor.In some embodiments, the data from annotated database includes and axialsignal values.

For example, if the desired application needs to detect if the person iswalking or running, data from an accelerometer could be used. Theannotated database will contain the timestamped raw data signals fromthe accelerometer, and an annotation label at each time to inform thesystem if the sensor signals correspond to walking or to running. Insome embodiments, the annotations are used to check that the machinelearning has the correct results.

In some embodiments, a user provides the raw data and the appropriateannotations. In some embodiments, there is an option for a user tosearch for what types of filters to apply to the raw data to extract thedesired features. In some embodiments, if there is initially a largenumber of features to explore then the machine learning system willreduce the number of features to the ones pertinent to the desiredapplication.

With the raw sensor data and annotations retrieved from annotateddatabase the performance optimizer will execute the desired applicationbased on the desired application and the plurality of raw sensor data.The performance optimizer will process the sensor data and extract atleast one feature from the sensor data for use in sensing at least oneactivity.

In some embodiments, the performance optimizer will analyze the rawsensor data with the purpose of generating results suited for thedesired application. In some embodiments, the performance optimizer willanalyze what features need to be extracted, the minimum number offeatures that should be extracted, what filters should be applied to theraw data to make it better represent the features (e.g., removingnoise), and the parameters and configurations required for the desiredapplication.

In some embodiments, the meta-analysis includes time and spectralanalysis, time frequency analysis, scripts, loss and cost functions,etc. In some embodiments, the performance optimizer will automaticallyapply filters to the data such as band pass filters, low pass filters,and high pass filters. In some embodiments, values such as mean,variance, peak to peak energy, dominant peaks, band power, distancebetween peaks, etc., are possible operations that can be applied to theraw sensor data. In some embodiments, the performance optimizer willanalyze if a sensor is important to the application. If a sensor isdeemed important, then the performance optimizer will analyze which axesof the sensor are most pertinent.

In one embodiment, the initial processing block requires nothing fromthe user but will analyze the data to identify aspects such as the valueof the cut off frequencies, what axis to use, if a sensor is needed(e.g., gyroscope), if all the known features need to be extracted or ifa subset is enough, etc.

In some embodiments, the performance optimizer analyzes what sensors arebeing used, which axes from the sensors are the most pertinent andcontain most valuable information that will help in generating thedesired application (for example, detecting if a person is driving,walking, running, speaking, what words are being spoken, etc.). In someembodiments, the performance optimizer analyzes what are thepreprocessing techniques and filters that will help clean the raw data,accentuate, and extract the most useful information for the desiredapplication. In some embodiments, the performance optimizer analyzeswhat features should be extracted to best represent the application,with the most valuable and pertinent amount of information to helpgenerate the desired application. In some embodiments, the performanceoptimizer analyzes what is the best classifier to maximize performance.

In some embodiments, the performance optimizer automatically selects asensor for use in the sensing of the desired application. In someembodiments, the performance optimizer automatically selects at leastone axis of the sensor for use in the sensing of the desiredapplication. In some embodiments, performance optimizer automaticallyselects the at least one feature to extract from the sensor data for usein sensing the at least one activity. In some embodiments, theperformance optimizer automatically selecting at least one preprocessingoperation to apply to the sensor data for extracting the at least onefeature. In some embodiments, the performance optimizer automaticallyselects at least one filter to apply to the sensor data for extractingthe at least one feature.

In some embodiments, the optimal data segment length and the inferencerate are variables that may be automatically selected by the performanceoptimizer. In some embodiments, the optimal data segment length is theamount of time over which a segment of data is collected from at leastone sensor, after which the segment of data goes to the ML system forpreprocessing and feature extraction. In some embodiments, the inferencerate is the amount of time between two decisions of the ML system. Insome embodiments, the decision made by the ML system involves collectinga new segment of data. For example, if an optimal data segment length isset to collect data for three seconds, with an inference rate of onesecond, then the two consecutive segments have an overlap of two second.

FIG. 7 illustrates an example output of the resource optimizer in theform of a table 700 showing the importance and cost of each feature,according to some embodiments.

After optimizing the performance of the desired application in the dataprocessing stage, the resource optimization stage assists in theunderstanding of how the desired application will run on the finalplatform. The goal of the resource optimization stage is to find abalance between the features included and the performance of thedeployed system.

In the resource optimization stage, an offline model is made to modelthe processor's memory, power, and MIPS usage in function of enabledprocessing blocks and generated models to be able to continuously keeptrack of those key elements when building the desired applicationoffline. In some embodiments, the offline model is made on a computersystem more powerful than the final platform. In some embodiments, theoffline model is made on a computer similar to training machine 101.

In some embodiments, the offline model will keep track of therequirements and make sure the desired application runs properly whendeployed to the embedded device. Based on the resource requirement forthe different features (e.g. in time or frequency, or time-frequencydomain), the best features can be selected, and the best featuresettings determined to get optimal performance based on sensorresources. The offline model can determine the required resources foreach feature block, and then select the best combination of features.This combination may be dependent on the application. The offline modelcan also determine which sensors to use, and how the adding or remove ofa sensor affects aspects such as performance, memory, cycles, and power.In some embodiments, based on these determined rules, the actual sensorcan set the sensor configuration.

In some embodiments, the offline model automatically selects the optimalfeatures and sensors to retain for the desired application. In someembodiments, a user can review the potential sensors and featuresthrough the resource optimizer.

Table 700 allows for a user to understand the cost of each feature whendeployed on the final platform (e.g. size, computation time, power,etc.). Table 700 also allows a user to see the relative importance of afeature. Using this table, a user could determine whether or not theywish to keep a feature active. In some embodiments, a user choosing todeactivate a feature is similar to deactivating a node in the decisiontree customization model 500. In some embodiments, the cost andimportance values are tallied or weighed into a single value, In someembodiments, the importance of a feature depends on the classifier anddesired application since it shows how important the feature is in theclassifying and identifying.

In some embodiments, the automatic processing chain generator isindependent of the classifier. In some embodiments, the automaticprocessing chain generator can be applied to systems with decisiontrees, neural networks, random forest, or any other classificationmethod. In some embodiments, the classifier and its parameters aresubjects that can be searched and optimized with the automaticprocessing chain generator.

Through the resource optimization phase, a user can keep the performanceof the device intact by running less complicated models and know theresulting performance change. The user is also able to understand howmuch of the system resources and power the desired application willconsume on the target platform, without having to deploy the model onthe platform.

For instance, from the results in table 700 a user might decide thatsensor 2 is unnecessary and might deactivate it. Alternatively, a usermight decide to keep features even at the cost of performance. In otherwords, a user is able to make tradeoffs in the structured processingchain. For example, if the user prefers having lower power (or cycles,or memory, or combination) at the expense of performance, the user canmake modifications to the solution processing chain to accomplish theirgoal before deploying the desired application to the target platform.

In some embodiments, the offline model tracks system requirements of atleast one feature for sensing at least one activity of the desiredapplication. In some embodiments, the system requirements include memoryconsumption for at least one sensor, computation consumption for atleast one sensor, and power consumption for at least one sensor. In someembodiments, the system requirements include memory consumption for atleast one feature, computation consumption for at least one feature, andpower consumption for at least one feature.

In some embodiments, the sensing of an activity utilizes a plurality offeatures. In some embodiments, a relative importance of the plurality offeatures used in sensing the at least one activity of the desiredapplication is determined. In some embodiments, the results of thesystem requirements for the at least one sensor for sensing the at leastone activity of the desired application are then aggregated.

In some embodiments, the results of the system requirements for theplurality of features for sensing the at least one activity of thedesired application are then displayed. FIG. 7 shows an example of howthe results can be displayed; however other formats are easilyapplicable.

With the results being displayed, in some embodiments a user reviews theresults and is able to select features or sensors for deactivation. Insome embodiments, after a feature or sensor is deactivated, the resultswill update to no longer include the deactivated feature or sensor andthe new results are displayed.

In some embodiments, a deselected feature is still displayed in theupdated results but has an indicator to show that the feature isinactive. In some embodiments, a separate set of inactive features andsensors is available for review.

In some embodiments, where resource limitations were included in thedesired application information, the offline model will automaticallyaccount for the resource limitations in the displayed results. In someembodiments, resource limitations are considered ideal but not required.In some embodiments, resource limitations are considered hard limits. Insome embodiments, a user can specify upon input if a resource limitationis a required limitation or not. In some embodiments, the automaticprocessing chain automatically disables features or sensors based on theresource limitations. In some embodiments, it is automaticallydetermined which features or sensors should be disabled based onresource limitations, but user confirmation is requested beforedisabling the feature or sensor.

FIG. 8 illustrates an example process for designing a processing chainof a sensor system, according to some embodiments. Procedures of thesemethods will be described with reference to elements and/or componentsof various figures described herein. It is appreciated that in someembodiments, the procedures may be performed in a different order thandescribed, that some of the described procedures may not be performed,and/or that one or more additional procedures to those described may beperformed. The flow diagrams include some procedures that, in variousembodiments, are carried out by one or more processors (e.g., a hostprocessor or a sensor processor) under the control of computer-readableand computer-executable instructions that are stored on non-transitorycomputer-readable storage media. It is further appreciated that one ormore procedures described in the flow diagrams may be implemented inhardware, or a combination of hardware with firmware and/or software.

At procedure 810, a desired application comprising at least one activityfor a sensor system to monitor is received. In some embodiments, thesensor system includes at least one sensor capable of generating sensordata based on sensing at least one activity. In some embodiments, the atleast one activity is a feature such as walking, running, sleeping, etc.In some embodiments, the at least one sensor is a sensor such as, forexample, an accelerometer, gyroscope, pressure, microphone, etc.

At procedure 820, a database including raw sensor data and annotationscorresponding to the raw sensor data is accessed. In some embodiments,the database is referred to as an annotated database. In someembodiments, the raw sensor data is a plurality of raw sensor data. Insome embodiments, the annotations are a plurality of annotations. Insome embodiments, the plurality of annotations identifies activitiescorresponding to the plurality of raw sensor data. In some embodiments,the raw sensor data includes timestamped data and axial signal valuesfor a plurality of sensors.

Next, starting at procedure 830, a processing chain is automaticallygenerated. The processing chain is of the sensor system for executingthe desired application based on the desired application and theplurality of raw sensor data. The processing chain will process thesensor data and extract at least one feature from the sensor data foruse in sensing the at least one activity.

At procedure 830, a sensor is automatically selected from the at leastone sensor to be used in the sensing of the desired application. In someembodiments, at least one axis of the sensor is automatically selectedfor use in the sensing of the desired application. In some embodiments,the optimal data segment length and inference length are automaticallyselected for use in the sensing of the desired application.

At procedure 840, at least one feature is automatically selected to beextracted from the sensor data for use in sensing the at least oneactivity. In some embodiments, at least one preprocessing operation isautomatically selected to be applied to the sensor data for extractingthe at least one feature. In some embodiments, at least filter isautomatically selected to be applied to the sensor data for extractingthe at least one feature. In some embodiments, the preprocessingoperation is used to reduce noise in the data and can be, for example, alow pass filter, a high pass filter, a bandwidth filter, etc. in someembodiments, the preprocessing operation is used to make the data betterrepresent the feature that is to be extracted.

At procedure 850, an offline model is created and the systemrequirements of the at least one feature for sensing the at least oneactivity of the desired application are tracked. In some embodiments,the offline model replicates the system requirements of the targetplatform. In some embodiments, the system requirements include memoryconsumption for the at least one sensor, computation consumption for theat least one sensor, and power consumption for the at least one sensor.In some embodiments, the system requirements include memory consumptionfor the at least one feature, computation consumption for the at leastone feature, and power consumption for the at least one feature. In someembodiments, the sensing of at least one activity utilizes a pluralityof features.

At procedure 860, the relative importance of the plurality of featuresused in sensing the at least one activity of the desired application isdetermined. In some embodiments, the system requirements for the atleast one sensor for sensing the at least one activity of the desiredapplication are then aggregated.

At procedure 870, the system requirements for the plurality of featuresfor sensing the at least one activity of the desired application aredisplayed. In some embodiments, a feature of the plurality of featurescan be selected for deselection from sensing the activity. In otherwords, a feature of the plurality of features can be deactivated. Afterat least one feature is deactivated, the displayed plurality of featuresis updated to remove the deselected feature from the plurality offeatures for sensing the activity. The updated system requirements forthe plurality of features for sensing the at least one activity of thedesired application is then displayed without considering the deselectedfeature.

In some embodiments, a deselected feature is still displayed in anupdated system requirements but has an indicator to show that thefeature is inactive. In some embodiments, a separate set of inactivefeatures is available for review.

In some embodiments, at least one resource limitation is received. Insome embodiments, the at least one resource limitation is received atthe start of the process. In some embodiments, the at least one resourcelimitation assists in automatically determining a relative importance ofthe plurality of features used in sensing the at least one activity ofthe desired application based on the at least one resource limitation.In some embodiments, a feature is automatically deselected based on theresource limitation.

CONCLUSION

The examples set forth herein were presented in order to best explain,to describe particular applications, and to thereby enable those skilledin the art to make and use embodiments of the described examples.However, those skilled in the art will recognize that the foregoingdescription and examples have been presented for the purposes ofillustration and example only. Many aspects of the different exampleembodiments that are described above can be combined into newembodiments. The description as set forth is not intended to beexhaustive or to limit the embodiments to the precise form disclosed.Rather, the specific features and acts described above are disclosed asexample forms of implementing the claims.

Reference throughout this document to “one embodiment,” “certainembodiments,” “an embodiment,” “various embodiments,” “someembodiments,” or similar term means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment. Thus, the appearances of suchphrases in various places throughout this specification are notnecessarily all referring to the same embodiment. Furthermore, theparticular features, structures, or characteristics of any embodimentmay be combined in any suitable manner with one or more other features,structures, or characteristics of one or more other embodiments withoutlimitation.

What is claimed is:
 1. A method for designing a processing chain of asensor system, the method comprising: receiving a desired applicationcomprising at least one activity for a sensor system to monitor, thesensor system comprising at least one sensor capable of generatingsensor data based on sensing the at least one activity; accessing adatabase comprising a plurality of raw sensor data and a plurality ofannotations corresponding to the plurality of raw sensor data, theplurality of annotations identifying activities corresponding to theplurality of raw sensor data; and automatically generating a processingchain of the sensor system for executing the desired application basedon the desired application and the plurality of raw sensor data, theprocessing chain for processing the sensor data and for extracting atleast one feature from the sensor data for use in sensing the at leastone activity.
 2. The method of claim 1, wherein the raw sensor datacomprises timestamped data and axial signal values for a plurality ofsensors.
 3. The method of claim 1, wherein the automatically generatinga processing chain of a sensor system for executing the desiredapplication based on the desired application and the plurality of rawsensor data comprises: automatically selecting a sensor of the at leastone sensor for use in the sensing of the desired application.
 4. Themethod of claim 3, wherein the automatically generating a processingchain of a sensor system for executing the desired application based onthe desired application and the plurality of raw sensor data furthercomprises: automatically selecting at least one axis of the sensor foruse in the sensing of the desired application.
 5. The method of claim 1,wherein the automatically generating a processing chain of a sensorsystem for executing the desired application based on the desiredapplication and the plurality of raw sensor data comprises:automatically selecting the at least one feature to extract from thesensor data for use in sensing the at least one activity.
 6. The methodof claim 5, wherein the automatically generating a processing chain of asensor system for executing the desired application based on the desiredapplication and the plurality of raw sensor data comprises:automatically selecting at least one preprocessing operation to apply tothe sensor data for extracting the at least one feature.
 7. The methodof claim 5, wherein the automatically generating a processing chain of asensor system for executing the desired application based on the desiredapplication and the plurality of raw sensor data comprises:automatically selecting at least filter to apply to the sensor data forextracting the at least one feature.
 8. The method of claim 1, whereinthe automatically generating a processing chain of a sensor system forexecuting the desired application based on the desired application andthe plurality of raw sensor data comprises: automatically selecting theoptimal data segment length and inference length.
 9. The method of claim1, further comprising: tracking system requirements of the at least onefeature for sensing the at least one activity of the desiredapplication.
 10. The method of claim 9, wherein the system requirementscomprise memory consumption for the at least one sensor, computationconsumption for the at least one sensor, power consumption for the atleast one sensor.
 11. The method of claim 9, wherein the sensing the atleast one activity utilizes a plurality of features.
 12. The method ofclaim 11, wherein the tracking system requirements of the at least onefeature for sensing the at least one activity of the desired applicationcomprises: determining a relative importance of the plurality offeatures used in sensing the at least one activity of the desiredapplication.
 13. The method of claim 12, wherein the tracking systemrequirements of the at least one sensor for sensing the at least oneactivity of the desired application comprises: aggregating the systemrequirements for the at least one sensor for sensing the at least oneactivity of the desired application.
 14. The method of claim 9, furthercomprising: displaying the system requirements for the plurality offeatures for sensing the at least one activity of the desiredapplication.
 15. The method of claim 14, further comprising: receiving aselection of a feature of the plurality of features for deselection fromsensing the activity. updating the plurality of features to remove thedeselected feature from the plurality of features for sensing theactivity; and displaying the system requirements for the plurality offeatures for sensing the at least one activity of the desiredapplication without considering the deselected feature.
 16. The methodof claim 9, wherein the system requirements comprise memory consumptionfor the at least one feature, computation consumption for the at leastone feature, power consumption for the at least one feature.
 17. Themethod of claim 9, wherein the tracking system requirements of the atleast one feature for sensing the at least one activity of the desiredapplication comprises: receiving at least one resource limitation;determining a relative importance of the plurality of features used insensing the at least one activity of the desired application based onthe at least one resource limitation; and automatically deselecting atleast one feature based on the at least one resource limitation.